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CROSS-REFERENCE TO RELATED APPLICATIONS 
[0001] This non-provisional application claims priority to US Provisional Patent 
Application Serial No. 60/208,659, filed on May 31, 2000, which is incorporated herein 
by reference. 
[0002] 

TECHNICAL FIELD 

The present invention relates to systems and methods for streaming media (e.g., 
video and audio) over a network, such as the Internet. 

BACKGROUND OF THE INVENTION 
[0003] With the success of the Internet and growing network resources, audio and video 
streaming is of enormous interest among Internet multimedia applications. Recently, tools 
such as RealPlay software and Microsoft NetShow® software have been developed to 
stream media content over a network. There remain, however, challenges to the streaming 
of media over the IP-based Internet due to issues such as the lack of a quality of service 
(QoS) guarantee, bandwidth variations, packet-losses, delays varying from time to time, 
and unknown network characteristics. 

[0004] Since the Internet is a shared environment and does not micro-manage utilization 
of its resources, end systems are expected to be cooperative by reacting to congestion 
properly and promptly. As a result, overall utilization of the network remains high while 
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each flow obtains a fair share of resources. Unfortunately, many of the current 
commercial streaming applications do not behave in a network-friendly fashion. 
[0005] The available bandwidth in the Internet fluctuates frequently in nature. Most 
conventional streaming applications are unable to perform quality adaptation as available 
bandwidth changes, especially quality adaptation among multiple streams. Thus, these 
conventional streaming applications do not make effective use of the bandwidth. 
[0006] To date several schemes have been developed for QoS management such as 
resource reservation, priority mechanism, and application level control. Prior art QoS 
management schemes and other background information, referred to elsewhere in the 
document, are presented in the following publications, each of which is incorporated 
herein by reference in entirety: 

[0007] 1 . R. Braden, L. Zhang, S. Berson et al, "Resource ReSerVation Protocol 
(RSVP) Version 1 Functional Specification", RFC 2205, September 1997 ("hereinafter, 
"Braden et al."); 

[0008] 2. R. Rejaie, M. Handley, and D. Estrin, "Quality adaptation for congestion 
controlled video playback over the Internet", Proceedings of SIGCOM99 ("hereinafter, 
"Rejaie et al [SIGCOM]")l 

[0009] 3. R. Rejaie, M. Handley, and D. Estrin, "An end-to-end rate-based 
congestion control mechanism for realtime streams in the Internet", Proceedings of 
INFOCOMM99, 1999 ("hereinafter, "Rejaie et al. [INFOCOMM\")\ 
[0010] 4. T. Chiang and Y. Q. Zhang, "A new rate control scheme using quadratic 
rate-distortion modeling", IEEE Trans. Circuits Syst. Video Technol.,Feb. 1997 
("hereinafter, "Chiang et al."); 
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[0011] 5. D. Sisalem and H. Schulztinne, "The loss-delay based adjusted algorithm: 
A TCP-friendly adaptation scheme", Proceedings ofNOSSDAV98, 1998 ("hereinafter, 
"Sisalem et al."); 

[0012] 6. J. Padhye, V. Firoiu, D. Towsley and J. Kurose, "Modeling TCP 
throughput: A simple model and its empirical validation", Proceedings of SIGCOMM'98, 
1998 ("hereinafter, "Padhye et al."); 

[0013] 7. O. Verscheure, P. Frossard and M. Hamdi, "MPEG-2 video services over 
packet networks: joint effect of encoding rate and data loss on user-oriented QoS", 
Proceedings ofNOSSDAV98, 1998 ("hereinafter, "Verscheure et al."); 
[0014] 8. A. Vetro, H. F. Sun and Y. Wang. "MPEG-4 rate control for multiple video 
objects". IEEE Trans. Circuits Syst. Video TechnoL, Feb. 1999 ("hereinafter, "Vetro et 
al."); and 

[0015] 9. M. Eckert and J. I. Ronda. "Bit-rate allocation in multi-object video 
coding". ISO/IEC JTC1/SC29/WG11 MPEG98/m3757, Dublin, Ireland ("hereinafter, 
"Eckert et al."). 

[0016] Among the foregoing QoS management schemes, resource reservation for 
supporting a certain QoS level, which was proposed by Braden et al., is the most 
straightforward approach. However, since it is difficult to know the characteristics of a 
stream in advance, one may tend to over-allocate resources in order to guarantee the 
requested QoS level, leading to network under-utilization. Besides that, the most 
challenging issue for the resource reservation mechanism is that it is both difficult and 
complex to implement and to deploy. 
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[0017] In priority mechanisms, different data packets or streams are labeled with different 
priorities and thereby treated differently at the network routers. While this approach is 
simple, the exact mechanism for setting the priority levels, the router mechanism for 
controlling these levels, and the actual gain are unclear. 

[0018] In application level control scheme, the QoS is controlled by adapting the sender 
transmission rate as was taught by Rejaie et al. [SIGCOM] and Chiang et al. Most of the 
control algorithms, however, randomly distribute resources among multiple streams 
without a global coordination mechanism. In order to employ a global coordination 
scheme, traffic control is usually adopted. There are several TCP-friendly rate adjustment 
protocols that have been reported recently. It has been proposed that transport protocols, 
including those taught by Rejaie et al [INFOCOMM] and Sisalem et al. who teach 
transport protocols where the throughput of a long-lived TCP connect is calculated based 
on the TCP characterization. However, Padhye et al. demonstrated that the above 
approaches to calculating the throughput are not accurate in cases where the packet-loss 
rate is higher than five percent (5%). Since this approach does not account for 
retransmission timeouts, it usually overestimates the throughput of a connection as the 
packet-loss rate increases. 

[0019] Other challenges to streaming video include network bandwidth adaptation, media 
adaptation, and error resilience. Network bandwidth adaptation deals with dynamic 
network bandwidth estimation, while media adaptation controls the media bit rate 
according to the network conditions. Error resilience refers to the ability to track, localize 
and recover transmission errors. 
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[0020] Without bandwidth adaptation, video transmission tends to compete unfairly with 
other TCP traffic, causing network congestion and resulting in a lower transmission rate 
for other TCP traffic. In addition, congestion collapse occurs when the aggregate 
bandwidth of the media traffic exceeds the network capacity. In order to dynamically 
adjust the transmission rate while co-existing with other TCP-based applications, several 
congestion control protocols have been proposed to adapt the sending rate in such a way 
that congested bandwidth is shared "fairly" with TCP applications. The majority of these 
protocols are based on TCP characterizations. Specifically, in the absence of 
retransmission time-outs, the steady state throughput of a long-lived TCP connect is given 



where C is a constant that is usually set to either 1 .22 or 1 .3 1 , depending on whether the 
receiver uses delayed acknowledgments, 7? is the round trip time experienced by the 
connection, and p is the expected number of window reduction events per packet sent. 
Since Equation (1) does not account for retransmission timeouts, it usually overestimates 
the connection throughput as packet-loss rate increases. It is has been reported that 
Equation (1) is not accurate for packet-loss rates higher than 5%. 
[0021] MPEG-4 is an object-based coding standard in which a visual scene typically has 
several video objects (VOs), each characterized by its shape, motion, and texture. The 
VOs are coded into separate bit streams that can be individually accessed and 
manipulated. The composition information is sent in a separate stream. To date several 
prior art rate control algorithms have been proposed. One such rate control algorithm was 
proposed by Chiang et al. for a single VO using a quadratic rate-quantizer model as the 
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Throughput = 



C 
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baseline rate control scheme used in the MPEG4 standard. This rate control scheme was 
extended by Vetro et al. to multiple video objects (MVOs). As taught by both Chiang et 
al and Vetro et al., the total target bit rate for all objects is controlled by a "joint-buffer" 
and allocated proportionally to the motion, size, and square of MAD (mean absolute 
distortion). For MVOs, Eckert et al. taught several straightforward approaches based upon 
the video object importance level. Although these approaches allow different objects to be 
encoded at different frame rates, when put in one scene, these objects with different frame 
rates may cause a break in the composition information and can result in unacceptable 
video reconstruction. 

[0022] When MPEG-4 video is transported over the Internet, all the above rate control 
schemes could not work well since they do not adapt to network bandwidth and packet- 
loss conditions that vary from time to time. Thus, the available resources could not be 
efficiently utilized and sometimes they may suffer from heavy congestion. 
[0023] It would be an advance in the art to devise a multimedia streaming TCP-friendly 
transport protocol that can adaptively estimate the network bandwidth and smooth the 
sending rate. It would also be an advance in the art to devise a global resource allocation 
control mechanism that maximizes the quality of AV streams delivered across fairly 
congested connections, where bits are allocated dynamically according to the media 
encoding distortion and network degradation. With respect to multiple video objects, it 
would be an advance in the art to devise a rate control scheme that uses such a multimedia 
streaming TCP-friendly protocol while minimizing the overall distortion under the 
constraint that the total rate for all objects is upper-bounded by a target bit rate. Finally, 
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an advance in the audiovisual streaming art would be achieved by minimizing the end-to- 
end distortion for a given network traffic condition and picture quality requirement. 
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BRIEF SUMMARY OF THE INVENTION 
[0024] The resource allocation architecture allocates resources among multiple media 
streams over a network (such as the Internet) to achieve optimized end-to-end quality of 
service. Examples of the media include audio, MPEG/H.26x compressed video, mesh, 
compressed image, and so forth. The resource allocation architecture copes with packet 
drops and fluctuating available bandwidth. From the network congestion control 
viewpoint, the architecture allows a multimedia streaming TCP-friendly protocol, which is 
suitable for multimedia streaming. From the visual quality control viewpoint, the 
architecture provides a way to maximize the visual quality for all streams based on the 
variable available bandwidth and the characteristics of each of the multiple media streams. 
The resource allocation architecture thereby enables the user to gain globally optimized 
quality at the client side. 

[0025] A method is disclosed for transmitting a mixed media data stream, including audio 
and multiple video objects (MVOs), between a sender and a receiver through a connection 
over a network. The transmission characteristics of the connection between server and 
receiver are monitored and the available bandwidth is estimated at the sender based upon 
the transmission characteristics of the connection monitored at the receiver side. A global 
buffer is allocated for the mixed media data stream to be transmitted from the sender to 
the receiver as a function of the estimated available bandwidth at the sender. A portion of 
each Video Object Plane (VOP) in the global buffer is pre-encoded with respect to a 
quantization parameter (QP) of the VOP. The VOP in the global buffer is then encoded 
based on the QP. An update is performed to a rate distortion model based upon the QP 
and a frame skipping function is performed after the VOP encoding. The sender then 
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transmits the encoded video object plane in the global buffer at a regulated sender 
transmission rate from the sender as a function of the estimated available bandwidth at the 
sender. 

[0026] In one implementation, a multimedia streaming TCP-friendly transport protocol is 
used to adaptively estimate the network bandwidth and smooth the rate at which a server 
sends streaming data to a receiver. A global resource allocation control mechanism is also 
implemented. This allocation control maximizes the quality of audio-visual (AV) streams 
delivered across congested connections, where bits are allocated dynamically to a global 
buffer for transmission over a network. This dynamic bit allocation is made in a manner 
that minimizes the media encoding distortion and network degradation. 
[0027] In an implementation dealing with multiple video objects, a rate control scheme 
uses a TCP-friendly protocol that minimizes the overall distortion under the constraint that 
the total rate for all objects is upper-bounded by a target bit rate. The TCP-friendly 
protocol is used to obtain network characteristics, including packet-loss rate, delay and 
jitter. Network characteristics are used to estimate available network bandwidth and to 
make adjustments to the sending rate in a smooth, non-erratic manner. The sending rate 
control scheme implemented herein allows the target bit rate and global buffer size to be 
dynamically adapted to the estimated available network bandwidth. Additionally, the 
packet-loss rate is taken into account when calculating the overall distortion. 
[0028] In each implementation, end-to-end distortion is minimized for network traffic 
conditions and picture quality requirements. 
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[0029] 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 depicts a general architecture where several continuous media servers 
play back AV streams for heterogeneous clients on demand over the Internet, where each 
server is able to support a large number of requests simultaneously, and where each client 
is able to request services from different servers. 

[0030] Figure 2 depicts an end-to-end framework for media streaming over the Internet. 
[0031] Figure 3 depicts the Gilbert Model which is able to capture the dependence 
between consecutive packet-losses, and wherein "1" is the received state and "0" is the 
loss state. 

[0032] Figure 4 illustrates a topology used in a simulation of an implementation of a 
resource allocation scheme. 

[0033] Figures 5 and 6 show simulation results of the throughput and sending rate, 
respectively, for six (6) different connections with their respective times. 
[0034] Figure 7 is a plot of peak signal to noise ratio (PSNR) per frame of different video 
objects, wherein the plot is illustrated with and without the disclosed resource allocation 
control scheme. 

[0035] Figure 8 is a plot of the PSNR per frame using two of the four objects in the 
MPEG-4 test sequence "Coast Guard", wherein the plot is illustrated with and without the 
disclosed resource allocation control scheme. 

[0036] Figure 9 shows a block diagram of the network resource allocation scheme 
disclosed herein for multiple video objects (MVOs). 
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[0037] Figure 1 0 shows the simulation results of the network available bandwidth and its 
estimation in a network conditions plot of bandwidth against time, wherein these two 
curves depict the real available bandwidth values and the estimated values obtained by 
using the multimedia streaming TCP-friendly transport protocol. 
[0038] Figures 11 and 12 both illustrate the disclosed network adaptive rate control 
scheme and the convention MPEG-4 Q2 scheme, where each show a plot of PSNR per 
frame, where Fig. 11 uses the MPEG-4 "coast guard" test sequence, and where Fig. 12 
uses the MPEG-4 "news" test sequence. 
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DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
[0039] The following discussion describes a system and method to allocate network 
resources among multiple media streams. These network resources are allocated within a 
client-network-server framework (such as the Internet) in a TCP-friendly manner to 
achieve optimized end-to-end quality of service. Simulation results are given that depict 
an implementation of the allocation of network resources to yield optimized end-to-end 
quality of service. A final discussion is then presented of an implementation of a 
network adaptive rate control scheme, including the results of a simulation of the same. 

[0040] 1. FRAMEWORK FOR INTERNET AUDIO-VISUAL STREAMING 

In media streaming, multiple servers and multiple clients are usually employed in 
the same session. Figure 1 depicts a general architecture where several continuous 
media servers play back AV streams for heterogeneous clients on demand, in which 
Server^ Server^ Server^ Server n are in communication with Clients Client p , Client q , and 
Client m through the Internet. Each server is able to support a large number of requests 
simultaneously and each client is able to request services from different servers. 
[0041] Figure 2 depicts an implementation of an end-to-end framework for media 
streaming over the Internet seen at reference numeral 220. End-to-end transport control 
is adopted by using a Multimedia Streaming TCP -Friendly transport Protocol (MSFTP) 
seen at reference numerals 212 and 222. The MSFTP protocol is a rate-based TCP- 
friendly protocol that continuously monitors the connection between sender and receiver, 
which are depicted in Figure 2, respectively, as Server 210 and Client 230. The results of 
the continuous monitoring of the connection are then used to regulate the bit transmission 
rate of Server 210. 
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[0042] A Quality of Service (QoS) Adaptation and Global Resource Allocation Control 
module 214 adjusts the quality of the bit stream that is transmitted from server 210 to 
client 230 through Internet 220. Module 214 periodically estimates the available 
bandwidth from the MSTFP protocol as derived from module 212. Combining this 
information with the media characteristics of media streams Video m? Video n , and Audiok, 
module 214 adjusts the quality of the total transmitted streams by resource re-allocation 
in a Global Buffer Control module 216 at Server 210. Data occupying the global buffer 
is transmitted from Server 210 in the MSFTP protocol at module 212 through Internet 
220 to Client 230. 

[0043] Client 230 processes the MSFTP protocol at module 232 and passes the media to 
a demultiplexer module seen in Figure 2 at reference numeral 234. Demultiplexer 
module 234 demultiplexes the combined stream into original media types for decoding at 
Videom decoder, Video n decoder, and Audiok decoder. The output from the video and 
audio decoders are mixed at a Media Mixer module 236 and output to an output device 
23 8 ? such as a personal computer at Client 230 having a display device and having a 
sound card with associated speakers. 

[0044] A discussion of the MSFTP protocol follows. Then, two exemplary 
implementations that use the MSFTP protocol are set forth. The first implementation is a 
resource allocation scheme for audio-visual streaming that is discussed in conjunction 
with the example illustrated in Figure 2. The second implementation is a network 
adaptive sending rate control scheme that is discussed in conjunction with the example 
illustrated in Figure 9. Both implementations will be discussed conceptually and in a 
presentation of results obtained by respective experimentations of each. 
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[0045] 2. MULTIMEDIA STREAMING TCP-FRIENDLY 

TRANSPORT PROTOCOL (MSTFP) 

To transmit the video and audio streams over the Internet, the characteristics of 

packet-loss and bandwidth fluctuation should be taken into account. The multimedia 
streaming TCP-friendly protocol (MSTFP) intends to minimize the number of future 
packets likely to be dropped and then smoothes the sending rate. Integral to the MSTFP 
protocol is the underlying TCP model taught by Padhye et al. which allows the sender to 
change its sending rate based on the observed packet-loss rate, round trip times, and 
retransmission time-outs. The two main characteristics of the MSFTP protocol are 
"TCP-friendliness" and "rate smoothness". 

[0046] The MSTFP protocol involves sender-side data packets and receiver-side control 
packets. The sender sends data packets to the receiver at a certain rate. The header of the 
sender-side packet includes the packet sequence number, a timestamp indicating the time 
when the packet is sent (S77), and the size of the sending packet. The receiver sends 
feedback to the sender at regular intervals. The receiver-side packet includes the time 
interval that the packet spent in the receiver side ( a RT), the timestamp of the packet sent 
from the sender (577), the estimated packet-loss rate, and the rate at which data is 
received. Based on the receiver's feedback, the sender uses the TCP model taught by 
Padhye et al. to adjust its sending rate in a TCP-friendly manner. 
[0047] What follows is a discussion of packet-loss rate estimation, round trip time 
estimation (RTT), retransmission time-out estimation (TO), available bandwidth 
estimation, and sending rate adjustment, each of which apply to the MSTFP protocol. 
The MSTFP protocol is preferred for the resource allocation implementation for audio- 
visual streaming and for the network adaptive sending rate control implementation. 
[0048] A. Packet-loss rate estimation 
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A two-state Markov Model (Gilbert model) is used to model the packet-loss in the 
network. Figure 3 depicts the Gilbert Model, wherein "1" is the received state and "0" is 
the loss state. This model is able to capture the dependence between consecutive losses. 

Network packets can be represented as a binary time series, j ? where x\ takes 1 if 

the ith packet has arrived successfully and 0 if it is lost. The current state, X x , of the 
stochastic process depends only on the previous value, X%.u The transition probabilities 
between the two states are calculated as follows: 

p = P[^ =11^=0] and q = P[X l =0\X l _ l =l]. (2) 

[0049] The maximum likelihood estimators of p and q for a sample trace are: 



A A 



p = n o\/ n o and ? = w 10 /v ( 3 ) 

where yiqi is the number of times in the observed time series when 1 follows 0 and n/o is 
the number of times when 0 follows 1. no is the number of 0s and n / is the number of 1 s 
in the trace. 

[0050] The probability of the event that the trace is in the loss state is given by: 

A 

p L =7r^- W 

p+q 

[0051] To obtain an accurate and stable measurement of the packet loss rate, the scheme 
uses a filter that weights the n (n = 8 in this implementation) most recent packet loss rate 
values in such a way that the measured packet loss rate changes smoothly. Thus, the 
estimated packet loss rate in the ith time interval is calculated as: 

n-\ 



J=0 (5) 
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where Lii ~ J is the measured packet loss rate in the (i-j)th time interval. 
[0052] It can be seen that the weighting parameters values, wj, used in calculating the 
packet loss rate determines the protocol's speed in responding to changes in the level of 
congestion. Two sets of weighting parameters are defined in this implementation, which 
are denoted as WS 1 and WS2, respectively. 
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[0053] In the above two parameter sets, WSl is used in the default cases. When the most 
recent packet loss rate is less than the half of the calculated packet loss rate, WS2 is 
adopted to concentrate more of the relative weight on the most recent packet loss rate. 
[0054] B. RTT and TO estimation 

Based on the receiver's feedback packet, the sender can adjust the current RTT 
using the following equation: 

RTT - a x RTT + (1 - a) x (now -STl-ART), (6) 

where RTT is the current round trip time, RTT is the estimated round trip time, now is 
the timestamp indicating the time at which the packet was received in the sender, and a is 
a weighting parameter that is set to 0.75 to smooth the variation for the purposes of this 
implementation. 

[0055] After calculating the RTT, retransmission timeout can be defined as: 



iee@hayes 



16 



MS1-589US.PAT.APP 



TO = RTT + (kx RTTVAR) , (7) 

where k is a constant that is recommended to take the value 4, RTT is the estimated round 
trip time of the sender packet from the sender to the receiver, and RTTVAR is a 
smoothed estimate of the variation of RTT. RTTVAR in Eq. (7) can be represented as: 



RTTVAR = a 2 x RTTVAR + (1 - a 2 ) x RTT - (now - STl - ART) . (8) 



In Eq. (8), RTTVAR is the current variation in the round trip time of the sender packet 
from the sender to the receiver (RTT), a2 is a weighting parameter that is preferably set 



to 0.75, and RTTVAR is a smoothed estimate of RTTVAR .. 



[0056] C. Available bandwidth estimation 

After the above procedures, the sender can estimate the present available network 
bandwidth as taught by Padhye et al. who calculate the present available network 
bandwidth as follows: 

PacketSize 

rcvrate = — , (9) 

R TT x fiPj3 + 3 x TO x P L x fipjs x (1 + 3 2P L 2 ) 

where PacketSize is the size of the sending packet and TO is a Time Out period that 
indicates the length of a time interval before which a sender should wait before re- 
transmitting a packet of data. As taught by Padhye et al., the present implementation 
proposes a TCP protocol that calculates TO without re-transmitting the data. 



[0057] D. Sending rate adjustment 

[0058] After estimating the available network bandwidth, the sender can dynamically 
adjust its sending rate. TCP congestion control is based on Additive Increase/ 



lee@hayes 



17 



MS1-589US PA TAPP 



I 1 

Multiplicative Decrease (AMD), which is independent of the lost fraction and adjusting 
interval. An alternate way to change the sending rate related to current packet-loss is 
described as: 
(10) 

if (rcvrate > currate) 

multi = (now - lastchange)/RTT 
constraint multi from 1 to 2 

currate = currate+ (PacketSize/RTT) x multi 

else 

y: 1 currate = J3 x rcvrate + ( 1 - J3) x currate , 

f s : where currate is the present sending rate, currate is the updated sending rate, lastchange 
p is the timestamp indicating the time at which last adjustment occurred, and is the 

s weighting parameter that is set to 0.75 in this implementation. An advantage of the above 

III scheme is that the sending rate can be increased dramatically to the available bandwidth, 
ill and in the meanwhile the rate can be decreased smoothly. In other words, the MSTFP 

protocol has less variation in the transmission rate and is less sensitive to random loss 

caused by channel error, 

[0059] 3, RESOURCE ALLOCATION FOR AUDIO-VISUAL STREAMING 

What follows is a discussion of an implementation of a resource allocation 
scheme for audio-visual streaming and a presentation of the results of simulations of the 
same. 

[0060] A, Discussion 
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Dynamic resource allocation is preferable for distributed multimedia systems that 
support application-level control. In the example illustrated in Figure 2, dynamic 
resource allocation can be implemented by integrating global buffer control 216 with 
Quality of Service Adaptation and Global Resource Allocation Control 216 using 
dynamic network bandwidth estimation. Dynamic network bandwidth estimations are 
made using the changing characteristics of Internet 220 learned at modules 212 and 232 
using the MSFTP protocol. One of the most challenging tasks of dynamic resource 
reallocation is global coordination of the feedback information from multiple streams. 
[0061] Different applications, such as file-transfer, web browsing, and AV 

streaming, have different tolerances that can be mismatched between the sending rate and 
the network bandwidth. For example, file-transfer has no real-time constraints and is 
tolerable to delay. The difference between the sensitivities to human aura and visual 
systems indicates the audio and video should be handled differently when adverse 
conditions arise, thereby affecting the playback of media streams. It is well known that 
the aural sense is more sensitive to disturbances than the visual sense. Therefore, it is 
appropriate to assign higher priority to audio data than video data. If data can be 
discarded when congestion occurs in the network, it is preferable to discard video data 
first. 

[0062] In general, the objective of resource allocation is to minimize the overall 
distortion under the total bit-rate constraint. If the sending rate of the ith media stream is 
denoted as r t , the distortion of \th media as d l , and the quality-impact parameter of this 

stream as a, , then a minimization of the distortion can be expressed as: 

Minimize D = £ cc. x d. , subject to R = s r < R T , (11) 

i i i 

i 
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where Rj is the total bit budget for the current time instant obtained from the MSTFP 
protocol, i.e., 



[0063] In the resource allocation scheme disclosed below, the main focus is on global 
buffer control and bandwidth allocation, where a global buffer is used to control the bit 
rate allocation. Using the MSTFP protocol, the current available network bandwidth Rj, 
can be estimated. Together with the number of bits spent in the global buffer in the 
previous time instant B prev , the size R 0 i/2 9 and the occupancy of the global output buffer 
W P rev in the previous time instant, the target rate and global buffer can be updated for 
each time interval. The output target rate of the joint buffer control is allocated among 
each stream to yield the target bit for an individual object. As the buffer size R 0 ic/2 is 
changed to Rt/2 9 the occupancy of the buffer is changed as follows: 



W cur -^( W vrev +B vrev> R T/ R old- R T/ F ^ °)» < 13 ) where "F" is the 



video frame rate (e.g., 20 frame/second, 30 frame/second, etc.) 

[0064] To achieve the global minimal distortion among the multiple streams, it is 

desirable to define the appropriate function for rate and distortion. 

[0065] To predict the number of bits that is used to encode the current stream for video 

objects, a quadratic function calculation is used that is taught by Chiang et al, and is 

given by : 



[0066] Verscheure et al. teach that packet-loss has an impact on video quality. For video 
distortion measuring, the present implementation considers the packet-loss that occurs 




(12) 



r. = 



p ll xMAD l p a K MAD l 



(14) 
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during video transmission. The video quality will drop quickly as the packet-loss rate 
and the sending rate go up. 

[0067] To take the packet-loss into account, the implementation uses another quadratic 
function to estimate the distortion that is given by: 

d l = q tl x Q t + q l2 x Q? + q l3 x r t x P Li . (1 5) 

[0068] In the audio case, one could still use the above distortion and rate control 
algorithms, respectively, in Eqs. (14) and (15). In Eq. (14), MADi is the energy for the 
audio case. 

[0069] Having the information of the rate and distortion, distortion can be minimized by 
Lagrange optimization to obtain the appropriate quantization parameters. These estimated 
quantization parameters are used in the encoding process. 
[0070] B. Simulation Results 

MPEG-4 streams with multiple video objects were used in a simulation to 
demonstrate the effectiveness of an implementation of the network resource allocation 
scheme for audio-visual streaming. MPEG-4 is an object-based video coding standard in 
which a visual scene is typically composed of Video Objects (VOs). Each VO is 
individually coded to give rise to an elementary bit stream that can be individually 
accessed and manipulated. The composition information is sent in a separate stream. For 
multiple video object streams, foreground objects receive most attention from the viewer, 
while background objects are of less importance. In addition, foreground objects usually 
change rapidly, while background objects change slowly. Therefore, they may have 
different impact on the overall video quality. 

[0071] In the simulation, the Network Simulator (NS) version 2 software was used to 
study the MSTFP protocol and the resource allocation mechanism for the MPEG-4 
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multiple video objects. The main tasks of the simulation were to select appropriate 
network topologies, effectively model the background traffic, and define proper 
performance metrics. 

[0072] Figure 4 is a topology used in the simulation. The topology has a single shared 
bottleneck link. The sources or senders are on one side of the link in Fig. 4 and the 
receivers are on the other side. All links except the bottleneck link are sufficiently 
provisioned to ensure that any drops/delays that occurred are only caused by congestion 
at the bottleneck link. All links are drop-tail links. In the simulation, the background 
traffic has infinite-duration TCP-like connections and infinite-duration real-time adaptive 
protocol (RAP) connections that were proposed by Rejaie et al. [SIGCOM] to deliver 
real-time traffic over Internet. 

[0073] As used herein, "friendliness" metrics are used as follows. k m denotes the total 
number of monitored MSTFP protocol connections and k t denotes the total number of 
monitored TCP connections. The throughputs of the MSTFP protocol connections are 

further denoted as T x m J™,„. y T™ and the throughputs of the TCP connections as 



7j',r 2 f . Then the average throughputs of the MSTFP protocol and TCP connections 
are respectively defined as follows: 

T M =^= LJ - and T T =^f^. (16) 

As such, the "friendliness" ratio can be defined as: F = T M /T T . (17) 
[0074] To verify "rate smoothness" for the MSTFP protocol connections, let 

R m ,R m 9 —,R m " represent the sending rates at different time instances 1, 2, . s m of the 



ith MSTFP protocol connection and , R? R?j represent those of the kth TCP 
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connection, then the sending variation of the MSTFP protocol connections and the TCP 
connections are respectively defined as: 




[0075] The smoothness ratio is defined as 

s = a mJ\. (19) 

[0076] By way of example, when S>1 then the kth TCP connection is smoother than the 
ith MSTFP protocol connection. 

[0077] Figure 5 shows the simulation results of the throughput as measured in Kb with 
respect to time measured in seconds for six (6) different connections. The data points for 
Figure 5 are listed in the Appendix. Two connections were used in an implementation of 
the MSTFP protocol (MSTPF1, MTPF2), two connections were the TCP protocol (TCP1, 
TCP2), and two connections (RAP1, and RAP2) were the Real time Adaptive Protocol 
(RAP) taught by Rejaie et al [SIGCOM]. The sending rates were measured in bps with 
respect to time. The six (6) different connections are illustrated in Figure 6. The data 
points for Figs. 5 and 6 are listed in the Appendix, below. From Figs. 5 and 6 it can be 
seen that the MSTFP protocol disclosed herein is more 'friendly 5 to TCP than RAP and 
also smoother than TCP and RAP. 

[0078] The simulation used the MPEG-4 test sequence "coast guard" to illustrate the 
performance of the resource allocation scheme. The coast guard test sequence includes 
four objects - river (objO), boat (objl), yacht (obj2), andriverbank (obj3). The four 
objects were coded in CIF at a temporal resolution of 10 fps. In the simulation, the 
quality-impact parameter for these objects was selected as 0.7, 1.2, 1.2, and 0.7, 
respectively. The bottleneck link bandwidth (kb) is set to 100, 140, 100, 80, 100,140, 
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100, 80, 100, and 140 to respectively correspond to time (ms) 0, 500, 1000, 1500, 2000, 
2500, 3000, 3500, 4500, and 5000. 

[0079] The test sequence plotted in Figure 7, the data points for which are listed in the 
Appendix, shows the PSNR per frame of different objects. The thick line illustrates the 
resource allocation control scheme and the thin line is an illustration without the network 
resource allocation control scheme. Fig. 7 illustrates coding at CIF 7.5 Frames Per 
Second (FPS) using the MPEG-4 test sequence "Coast Guard" but, for the benefit of 
clarity, illustrates only the simulation results of objl and obj3. 

[0080] Figure 8 shows plots of the PSNR per frame coded at CIF 7.5 Frames Per Second 
(FPS) and using the MPEG-4 test sequence "Coast Guard". The data points for Fig. 8 are 
listed in the Appendix, below. An implementation of the resource allocation control 
scheme is seen in a solid line and the dashed line illustrates a rate control scheme without 
this implementation. Note that there are several sharp drops in the resource allocation 
control scheme due to frame skipping. Furthermore, it can be seen that during the period 
of no frame skipping, the resource allocation control scheme has a higher PSNR. It can 
be seen in Figure 8 that, even taking the skipped frames into consideration, the 
implementation of the resource allocation control scheme achieves a better video quality. 
The data points for Figure 8 are listed in the Appendix, attached hereto. 
[0081] Table 1 shows a comparison of the results with and without the MSTFP protocol 
resource allocation scheme disclosed herein. The multiple video streams for the MPEG-4 
test sequence "coast guard" were used to produce the results seen in Table 1 for each of 
the four objects - river (objO), boat (objl), yacht (objl), and riverbank (obj3). 
[0082] Table 1 
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Resource 
control 
scheme 


ObjO 


Objl 


Obj2 


Obj3 


Average 
PSNR 


MSTFP 
Resource 
allocation 


W 


PSNR 


W 


PSNR 


W 


PSNR 


W 


PSNR 


24.16 


0.7 


24.64 


1.2 


19.52 


1.2 


19.91 


0.7 


23.99 


Without 
control 


PSNR 


PSNR 


PSNR 


PSNR 


20.99 


24.23 


15.45 


18.12 


22.30 



[0083] 4. NETWORK ADAPTIVE RATE CONTOL SCHEME 

An implementation of a network adaptive rate control scheme will now be 
presented in conjunction with the example illustrated in Figure 9. Also, the results of a 
simulation of an example of a network adaptive rate control scheme are set forth. 
[0084] A. Discussion 

The Network Adaptive Rate Control (NARC) scheme disclosed herein achieves 
minimal distortion in a global scene. Specifically, an overview is first given of an 
implementation of the NARC scheme, followed by a focus upon a quantizer adjustment 
of the NARC scheme. 

[0085] Figure 9 shows a block diagram of the NARC scheme for multiple video objects 
(MVOs). The block diagram depicts a flow chart for the calculation of an estimate of the 
MSTFP available bandwidth that is made at module 910 of Figure 9. 
[0086] Unlike the prior art rate control schemes taught by Chiang et al. and Vetro et al., 
implementations of the MVOs NARC scheme includes features such as dynamically 
estimating the available bandwidth at module 910, the target rate and global buffer 
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updating at module 912, quantization adjustment at module 934, and frame skipping 
control at module 960. Functionalities of these modules are discussed below. 
[0087] The MVOs NARC scheme illustrated in Fig. 9 includes five stages, each of which 
will now be described. In the first stage, illustrated generally at module 910, the current 
available network bandwidth (R T ) is estimated in accordance with the MSTFP protocol. 
Together with the number of bits spent in the previous time instant (B prev ), the size 
(Roi/2) and the occupancy (W prev ) of the encoder output buffer, the target rate and global 
buffer are then updated for each Video Object Plane (VOPi) frame at module 912. The 
output total target bits from the joint buffer control are allocated among each Video 
Object Plane (VOPi) at module 914 to yield the target bits for each individual object. As 
the buffer size, R 0 i/2, is changed to R T /2, the occupancy of the buffer, W cur , is changed as 
follows: 

W = max«TF + £ )xR /R -R lF\ 0 , (20) 
cur vv prev prev' Tl old Tl J v J 

where "F" is the video frame rate (e.g., 20 frame/second, 30 frame/second, etc.) 
[0088] In the second stage, which is the pre-encoding part seen generally at reference 
numeral 930, the quantizer of each VOPi is adjusted. Chiang et al. teach that the 
relationship between the quantization parameter {QP) and the texture value can be 
described as 

p. x MAD p x MAD 
r = OP + 2 > (21) 

QP Qp£ 

[0089] where pi and p 2 are control parameters. In the pre-coding part, initialization of 
the control parameters p^ and p^ takes place at initialization module 932. In order to 

maintain a constant quality of the total video sequence, the QP of each VOPi is limited to 
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the range from 1 to 3 1 , depending upon the bandwidth. The QP is allowed to change 
within a pre-defined range compared with the previous QP. A QP of 1 reflects the best 
quality and 3 1 represents the worst quality. 

[0090] The Quantization Adjustment module 934 performs functions that contribute 
towards a global reduction in distortion. To achieve the global minimal distortion among 
the MVOs, the implementation of the MVOs NARC scheme employs the quadratic 
function to predict the number of bits, r, , which will be used to encode the ith VOP: 

(p.YxMAD (p)xMAD. 

r ;=- L 7^ — L+ 2 (22) 

QP i QP? 

[0091] Still another quadratic function proposed by Chiang et al. is used to estimate the 
distortion, and is given by: 

* = x QP + (q 2 \ x QP? , (23) 
where qi and q 2 are control parameters. 

[0092] Verscheure et al. teach that packet-loss has an impact on video quality. A rate 
control implementation disclosed herein takes into consideration the packet-loss that 
occurs during video transmission. Video quality deteriorates quickly as the packet-loss 
rate and the sending rate go up. 

[0093] To take the packet-loss into account, Eq. (23) is modified to: 

d t = fo^. x QP + (q 2 ) i x QP? + (q 3 ) . x r. x (P £ ) . , (24) 
where q3 is the additional control parameter. 

[0094] The objective of a rate control is to minimize the overall distortion to obtain the 
optimized quality for the whole video frame. Such a minimization can be expressed as: 
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minimize D = J]d., subject to R = £r. < Hp , (25) 

I I 

where R T is the total bit budget for the current time instant obtained from the buffer 
control algorithm and the MSTFP protocol. The third stage, seen generally in Fig. 9 at 
reference numeral 940, performs an encoding process based upon the quantization 
parameter (QP) estimate from the second stage, above. The "Intra coding" at module 944 
receives the QP from pre-encoding Quantization Adjustment module 934 and is directed 
to the I-picture frame. The Quantization Adjustment module 934 is discussed further 
below. The "Inter coding" at module 942 receives initialized values from a pre-coding 
initialization module 932 and is directed to the P-picture frame or the predicted picture. 
Both "Intra coding" module 944 and the "Inter coding" at module 942 pass I-picture 
frames and P-picture frames, respectively, to the post encoding fourth stage seen 
generally at reference numeral 952 in Fig 9. 

[0096] In the fourth stage, the rate model is updated at module 954. The objective of a 
rate control is to minimize the overall distortion. The update to the distortion model is 
performed at module 956. The rate and distortion models are updated based on the 
encoding results of the current frame as well as those from a specified number of the 
previous frames. Both the rate and the distortion aspects of the rate and distortion models 
are functions of the quantization parameter (QP) from Eq. 21 and the probability that the 
trace is in the loss state ( ) from Eq. 4. The update to the rate and distortion models 

can be done using the least squares procedure described by Eckert et al. 
[0097] The fifth stage is a frame skipping module 960 in Fig. 9. In the fifth stage, frame 
skipping occurs after all the video objects (VOs) in the scene have been encoded. Frame 
skipping occurs at this point in the implementation because, when the global scene is 
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considered, all the VOs composed within the same scene should be encoded at the same 
frame rate. For the purpose of frame skipping, the P-picture frame has a lower priority 
and the I-picture frame has a high priority. As such, the P-picture frame will preferably 
be the frame that is chosen to be skipped. 
[0098] B. Simulation Results 

To illustrate the bandwidth adaptation ability of an implementation of the NARC 
scheme, a simulation was conducted of the MSTFP protocol using the Network Simulator 
(NS) software version 2. The results of the simulation are illustrated in Figure 10 and the 
data points therefore are listed in the Appendix. Fig. 10, discussed below, demonstrates 
that given a variable available bandwidth, the MSTFP protocol can dynamically adjust 
the sending rate to match up with the available network bandwidth. 
[0099] Figure 1 0 depicts the available network bandwidth and its estimation in plot of 
bandwidth measured in kbps against time measured in ns. The available network 
bandwidth is illustrated in Fig. 10 as a dashed line and the MSTFP estimated network 
bandwidth is illustrated by a solid line. 

[0100] In the simulation, a standard MPEG-4 codec was used with unrestricted motion 
vector modes. The simulation tested two rate control schemes: 1) the bandwidth adaptive 
MVOs rate control scheme disclosed herein, and 2) a conventional MPEG-4 standard 
MVOs rate control (Q2) without any feedback from network bandwidth and packet-loss 
rate. A change was made in the frame skipping part of the conventional MPEG-4 Q2 
scheme so as to maintain all the objects of the same scene in a consistent frame rate. In 
both cases, the first frame was intracoded, and the remaining frames were intercoded. The 
tests used the MPEG-4 "coast guard" video test sequence and the MPEG-4 "news" test 
video sequences, respectively. Both of them are composed of four video objects. 
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[0101] PSNR was used in the simulation as a metric to measure video quality. For an 
eight-bit image with intensity values between 0 and 255, the PSNR is given by: 



PSNR — 20log 1 A 



255 



RMSE 



(26) 



where RMSE stands for root mean squared error. Given an original NxM image/ and a 
compressed or degraded image / the RMSE can be calculated as follows: 



RMSE = 



I N-lM-1 0 

S £ lf(x 9 y)-f(x 9 y)r. (27) 



NxM x = Qy = 0 



[0102] A fair comparison of PSNR between the two rate control methods with the 
skipping of different frames is not trivial. A rate control technique that skips more frames 
would typically spend more bits per coded frame and could easily have a very high 
average PSNR per coded frame. Table 2, below, presents the rate control impact on 
video quality. The average PSNRs of the encoded frames are shown in the third column 
of Table 2. It can be seen from Table 2 that there is more frames skipping in the 
conventional MPEG-4 Q2 scheme than in an implementation of the NARC scheme. 
[0103] Table 2 



Video 


Rate 


Encoded 


Skipped frame 


Total average 


Sequence 


control 


average 




PSNR 




scheme 


PSNR 






Coast guard 


NARC 


26.02 


20 


23.92 


(~100kbps) 


scheme 










MPEG-4 


27.33 


26 


21.89 




Q2 










scheme 
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News 


NARC 


30.97 


8 


30.77 


(-100 kbps) 


scheme 










MPEG-4 


33.08 


25 


29 36 




Q2 










scheme 









[0104] Table 2 also demonstrates that the encoded average PSNR in the conventional 
MPEG-4 Q2 scheme is higher than in the NARC scheme. However, using the encoded 
average PSNR value as a measure of compressed video quality is not accurate. This is 
because the distortion of the non-encoded video is not taken into account. In the rate- 
control testing in the conventional MPEG-4 Q2 scheme, it was decided that when a frame 
was skipped, the previous encoded frame should be used in the PSNR calculation because 
the decoder displays the previous encoded frame instead of the skipped one. The average 
PSNR is calculated using this approach herein and the results are shown in Table 2 as the 
"TOTAL average PSNR" in column 5. 

[0105] Figs. 1 1 and 12 each show a plot of PSNR per frame at 10 frames per second, 
where Fig. 1 1 uses the MPEG-4 "coast guard" test sequence and Fig. 12 uses the MPEG- 
4 "news" test sequence. The respective data points for each of Figures 1 1 and 12 are 
listed in the Appendix. In Figures 1 1 and 12, the solid line illustrates an implementation 
of the NARC scheme and the dashed line illustrates the convention MPEG-4 Q2 scheme. 
The PSNR values of the tested video drops sharply due to frame skipping. Figs. 1 1 and 
12 illustrate that the NARC scheme significantly reduces the number of frames that are 
skipped. During the period when no frames are skipped in either method, the NARC 
scheme has a higher PSNR, and therefore video quality, than that of the conventional 



iee@hayes 



MS 1-589 US PA TAPP 



MPEG-4 Q2 scheme. This is because the packet-loss rate in the NARC scheme is less 
than that in the conventional MPEG-4 Q2 scheme. 

[0106] The above-described implementations of systems and methods are presented 
herein in the general context of server-client architecture via the Internet. The client and 
server may be implemented by general-purpose computers, or by other particularly 
tailored computing devices, such as minicomputers, mainframe computers, workstations, 
Internet appliances, set top boxes, wireless communication devices, game consoles, 
portable computing devices, and the like. 

[0107] An exemplary computer system has one or more processors, memory, and one or 
more input/output (I/O) devices. The processor communicates with the memory and I/O 
devices via one or more bus/interface mechanisms. Programs, data, files, and the like 
may be stored in memory and executed on the processor. 

[0108] The memory typically includes both volatile memory (e.g., RAM) and non- 
volatile memory (e.g., ROM, Flash, hard disk, CD-ROM, DVD, etc.). Examples of 
possible I/O devices include a mouse, a touch pad, a joystick, a keyboard, a keypad, a 
display, a peripheral storage device, and a microphone. The computer system may also 
implement a computer operating system that is stored in memory and executed on the 
processor. 

[0109] The present invention may be embodied in other specific forms without departing 
from its spirit or essential characteristics. The described embodiments are to be 
considered in all respects only as illustrative and not restrictive. The scope of the 
invention is, therefore, indicated by the appended claims rather than by the foregoing 
description. All changes which come within the meaning and range of equivalency of the 
claims are to be embraced within their scope. 
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[01 10] APPENDIX 

Fig. 5 - Six (6) plots of sixty (60) data points, from 1 to 60; 
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[0111] Fig. 6 - Six (6) plots of sixty (60) data points, from 1 to 60: 
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[0112] Figure 7 - Four (4) plots of thirty-eight (38) data points, from zero to 37: 

Obil (No) 

30.37 25.3 19.92 19.9 19.62 19.4 20.99 18.82 17.36 17.09 16.66 

16.93 15.31 14.11 13.97 13.69 13.73 14.45 13.44 12.57 12.58 12.69 

12.71 12.77 12.8 12.89 13.12 13.13 13.09 13.05 12.95 12.88 12.9 
13 13.11 13.16 13.26 13.37 

Obi3 (NO) 

31.46 30.98 30.2 29.81 29.6 29.28 29 28.7 28.53 28.04 27.51 

27.23 25.43 23.75 23.27 22.87 22.46 22.82 17.6 16.41 16.44 16.58 

17.16 17.37 17.37 17.45 17.65 17.67 17.55 17.74 17.92 18.18 18.3 
18.49 18.9 19.14 19.26 19.44 

Objl 

30.37 8.51 7.03 6.7 25.11 24.35 24.23 23.78 23.82 23.72 24.21 

24.07 24.21 8.68 24.23 8.56 6.75 24.88 4.09 24.33 9.98 24.09 

24.03 23.99 24.02 23.84 23.83 24.22 12.42 24.23 11.26 9.13 24.08 
14.88 23.94 23.94 24.14 24.26 

Obi3 

31.46 20.79 19.94 19.66 26.47 27.35 27.34 27.49 27.54 27.61 27.73 

27.83 27.15 18.87 26.8 18.61 17.57 27.79 14.97 25.51 18.73 26.06 

24.98 25.62 25.9 25.97 26.12 25.23 16.96 25.85 16.74 16.19 25.76 
18.34 25.93 26.38 26.4 26.2 



[0113] Figure 8 - Two (2) plots of thirty-eight (38) data points, from zero to 37: 

With Resource Allocation 

31.38 22.81 21.44 20.69 26.68 26.88 26.82 26.7 26.68 26.61 26.67 

26.66 26.43 19.71 26.3 19.18 16.99 27.67 15.57 25.99 18.18 26.17 

25.51 25.82 26.02 26 26.2 25.9 18.1 26.2 17.28 16.53 26.21 
20.29 26.24 26.58 26.64 26.57 

Without Resource Allocation 

31.36 28.04 27.36 26.93 26.79 26.53 26.61 25.88 24.87 24.46 23.97 

23.81 22.46 21.37 20.97 20.68 20.54 21.5 17.26 16.28 16.32 16.58 

16.97 17.12 17.22 17.32 17.64 17.8 17.51 17.61 17.66 17.88 18.08 
18.3 18.64 18.88 19.04 19.34 
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[0114] Figure 10 - Two (2) plots of fifty (50) data points, from 0.0 to 4.9, 
in increments of 0.1: 
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MSTFP Available Bandwidth 
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[0115] Figure 11 - Two (2) plots of fifty (50) data points from 1 to 50. incrementing by 1: 
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MPEG-4 02 Scheme 










31.36 


28.38 


28.1 


28.07 


27.91 


27.9 


27.79 


27.74 


27.81 


27.65 


27.39 


27.16 


27 


27.04 


27.07 


25.62 


25.62 


20.54 


24.41 


20.06 


18.45 


17.23 


25.32 


17.12 


15.14 


14.1 


26.83 


20.86 


17.87 


17.39 


16.8 


16.41 


15.96 


25.21 


19.56 


17.48 


16.24 


15.84 


15.49 


22.82 


17.85 


17.05 


16.67 


16.53 


23.91 


19.12 


17.36 


16.88 


16.89 


20.37 













lee@hayes 



36 



MS1-589US PA TAPP 



[0116] Figure 12 - Two (2) plots of seventy-four (74) data points, from 1 to 74, 
incrementing by 1; 



NARC Scheme 

34.76 28.22 26.89 25.72 25.38 25.11 24.65 24.34 24.06 33.68 33.41 

32.95 32.64 32.35 32.26 32.18 32.04 31.93 32.03 32.04 32 31.94 

31.97 31.81 31.94 31.85 31.93 31.86 31.76 31.67 31.65 31.51 31.6 

31.65 31.64 31.51 31.48 31.49 31.35 31.25 31.14 31.03 31.06 30.86 

30.84 30.85 30.61 30.69 30.76 30.78 30.93 30.84 30.84 30.75 30.79 

30.56 30.61 30.66 30.63 30.75 30.73 30.75 30.83 30.78 30.75 30.55 

30.63 30.59 30.77 30.78 30.65 30.74 30.93 30.83 30.9 



MPEG-4 Q2 Scheme 

34.76 28.22 26.89 25.72 33.93 33.58 33.06 30.85 32.62 32.6 28.08 

32.29 32.32 27.83 32.08 28.03 31.83 31.84 31.87 31.9 31.83 27.72 

31.83 31.61 31.6 31.59 31.52 31.41 31.44 31.34 31.31 27.39 31.25 

31.4 31.5 31.36 27.19 31.26 31.32 28.8 31.08 31.07 27.56 30.88 

27.53 30.93 22.66 30.58 25.03 30.66 23.99 30.76 26.52 27.71 24.92 

27.39 23.9 27.13 27.5 28 28.16 25.83 28.45 28.41 28.57 24.52 

28.74 24.63 29.18 29.35 23.48 29.41 24.99 30.01 27.29 
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